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1.  INTRODUCTION 

\ 

1.1  Purpose.  This  report -sjias  been  prepared  in  response  to 
Defense  Communications  Agency  (DCA)  Task  Order  6-83,  issued  under 
the  terms  and  conditions  of  Contract  DCA100-78-C-0053  as  amended. 

It  constitutes  Contract  Data  Requirements  List  (CDRL)  Line  Item 
001  of  the  Task  Order  .'^In,  accordance  with  the  Statement  of  Work 

\  i  f  i  V*  *  - 

(SOW) ,  its  purpose  is  to  i-dentify  all  Defense  Data  Network  (DDN) 
testable  components  (hardware,  software),  assemblies,  subsystems, 
integrated  facilities,  and  subsystems;  to  describe  the  specific 
nature  and  objective  of  the  tests  required  to  assure  proper 
network  performance,  including  recommended  schedules  and 
locations;  and  to  recommend  which  software  and  firmware 
developments  should  be  monitored  by  Independent  Verification, 
Validation,  and  Test.  (IW&T) . 

1.2  Objective.^-The* objective  of  this  report  is  to  provide  to  the 
Government,  in  an  easily  accessible  form,  information  needed  for 
the  development  of  a  Test  and  Evaluation  Master  Plan  (TEMP)  for 
the  DDN^  The  requirement  for  a  TEMP  and,  therefore,  for  the 
information  contained  in  this  report,  stems  from: 

(*)  The  need  to  perform  test  and  evaluation  on  DDN  component 
/  elements  and  subsystems  to  assure  proper  network 
/  performance. 

/  (b)  The  need  for  IW&T  of  the  critical  software  and  firmware 

elements  of  the  DDN. 

(c)  The  need  to  develop  general  descriptions  and  objectives 
of  performing  Test  and  Evaluation  (T&E)  and  IW&T  actions. 

This  report  provides  the  information  necessary  to  meet  these 
requirements.  Its  content  is  in  accordance  with  the  requirements 
of  Subtask  1  of  the  Task  Description  of  Task  Order  6-83. 

1.3  Background.  The  DDN,  when  fully  developed,  will  be  an 
integrated,  packet-switching  network  serving  the  Department  of 


1-1 


Defense  (DoD).  The  DDN  is  made  up  of  two  functional  areas:  (1) 
the  network  backbone,  which  is  based  on  Advanced  Research  Projects 
Agency  Network  (ARPANET)  packet-switching  technology  and  includes 
the  trunk  circuits  and  packet  switches,  and  (2)  the  subscriber 
access  network,  which  will  enable  the  subscriber,  through  the  use 
of  circuits  and  interface  equipment,  to  connect  to  the  backbone. 

The  DDN  backbone  will  contain  approximately  200  packet  switches  at 
100  sites.  The  hardware  for  the  switches  is  the  C/30 
microprogrammed  minicomputer,  produced  by  Bolt,  Beranek  &  Newman, 
Inc.  (BBN) .  It  will  be  compatible  with  existing  ARPANET  interface 
message  processor  (IMP)  packet-switching  software.  Most  backbone 
circuits  will  be  leased  landlines.  They  will  be  either  digital 
(56  kbps)  or  analog  (9.6  kbps  overseas  and  50  kbps  in  the 
continental  U.S.  (CONUS)).  Links  to  overseas  sites  will  be 
through  satellite. 

To  access  the  network,  subscriber  hosts  will  connect  to  the 
backbone  packet  switches  through  Host  Front  End  Processors  (HFEPs) 
or  Terminal  Emulation  Processors  (TEPs) ,  or  directly  through  a 
portal  on  the  host  computer.  These  possible  connections  are 
illustrated  in  Figure  1-1.  Host  circuit  transmission  speeds  will 
be  2.4  kbps  to  56  kbps.  Hosts  requiring  high  network  availability 
will  be  able  to  connect  to  more  than  one  packet  switch  through 
multiple  circuits.  Terminal  connectivity  into  the  network  will  be 
by  way  of  a  Terminal  Access  Controller  (TAC)  or  indirectly  by 
routing  through  a  host  which  is  directly  connected  to  the 
backbone.  TACs,  using  either  leased  or  dial-up  lines,  will  accept 
up  to  64  terminals.  Line  speeds  will  vary  from  100  bps  to  19.2 
kbps.  Each  TAC  will  enter  the  backbone  through  a  packet  switch  by 
way  of  a  4.8  kbps  or  56  kbps  line. 

Four  safeguards  will  protect  network  security.  First,  Internet 
Private  Line  Interfaces  (IPLIs)  will  be  used  to  provide  end-to-end 
encryption  and  to  separate  the  traffic  by  each  level  of  security 
or  by  special  communities  of  interest.  Second,  traffic  flow 


Figure  1-1.  Subscrib 
Connectivity  to  DDN 


security  will  be  provided  by  link  encryption  used  on  the  backbone 
trunks  and  access  lines  where  required.  The  third  measure  is 
locating  packet  switches  and  TACs  in  restricted  access  areas.  The 
fourth  measure  will  be  the  TEMPEST  certification  of  all  C/30 
packet  switches  and  TACs. 

DON  operational  status  control  will  be  maintained  through  network 
Monitoring  Centers  (MCs) .  The  MCs  will  conduct  fault  diagnosis 
and  provide  software  maintenance  for  DDK.  Currently  it  is  planned 
to  have  a  primary  MC  and  designated  alternate  MCs  in  the  United 
States,  with  regional  MCs  in  Europe  and  the  Pacific.  MCs  will 
also  be  provided  for  each  separately  keyed  subscriber  community. 
The  MC  will  consist  of  one  or  more  BBN  C/70  minicomputers  using 
the  current  ARPANET  monitoring  center  NU  software. 

Development  of  the  DON  will  take  place  in  four  stages  as  reflected 
in  the  Program  Plan.  The  first  stage  includes  the  ARPANET  split 
into  the  Military  Network  (MILNET)  and  the  Experimental  ARPANET, 
and  integration  of  the  Movements  Information  Network  (MINET)  into 
MILNET.  The  second  stage  involves  Network  Access  Component  (NAC) 
availability  for  terminal  access  to  DDN.  The  third  stage  involves 
integration  of  the  IPLI  for  community  of  interest  separation  in  an 
integrated  network.  The  fourth  stage  occurs  upon  introduction  of 
Blacker  technology  cryptographic  devices.  During  stages  two  and 
three,  other  networks  will  be  merged  to  form  the  fully  developed 
DDN.  Also,  improvements  and  enhancements  will  be  made  as  the 
system  develops  to  meet  the  needs  of  more  and  more  subscribers. 
Subscribers  will,  therefore,  be  added  during  each  stage  of 
development. 

1.4  Scope .  Although  the  development  of  the  DDN  does  not  follow 
the  standard  acquisition  process  with  a  T&E  program  structure  to 
support  that  process,  a  well  reasoned  test  program  is  required  and 
attainable.  The  Defense  Data  Network  Program  Plan  of  January  1982 
(Revised  May  1982)  gives  the  T&E  Division  of  the  DDN  Program 
Management  Office  (PMO)  responsibility  for  three  levels  of  testing 


for  the  DDN.  The  first  is  developmental  testing,  the  second  is 
initial  operational  testing,  and  the  third  is  full  operational 
testing.  The  draft  Defense  Data  Network  Management  Engineering 
Plan  (MEP) ,  dated  December  1982,  describes  three  basic  testing 
situations  which,  among  them,  include  the  three  levels  of  testing 
mentioned  in  the  Program  Plan.  They  are: 

(a)  Testing,  at  the  component  level,  of  new  or  modified 
components  (hardware  and  software)  to  verify  that  they 
perform  in  accordance  with  specifications.  This  includes 
testing  of  interfaces  between  two  or  more  components. 

(b)  Testing  of  the  equipment  at  a  switching  node  or 
subscriber  site  when  that  site  is  added  to  the  network. 

(c)  Testing  which  is  necessary  when  a  network  is  integrated 
into  the  DDN. 

These  situations  include  all  necessary  developmental  and 
operational  testing  and  provide  a  framework  for  identifying 
required  DDN  testing.  This  report,  therefore,  uses  this 
three-part  breakdown  in  describing  appropriate  T&E  and  IW&T 
actions  for  the  DDN.  It  is  not  within  the  scope  of  the  report  to 
describe  testing  procedures  or  to  detail  the  testing  methods  to  be 
employed.  This  information  is  appropriately  included  in  test 
plans. 

In  accordance  with  the  requirements  of  Department  of  Defense  (DoD) 
Directive  5000.3,  recommendations  for  Developmental  Test  and 
Evaluation  (DT&E) ,  Initial  Operational  Test  and  Evaluation 
(IOT&E),  Follow-on  Test  and  Evaluation  (FOT&E) ,  and  Production 
Acceptance  Test  and  Evaluation  (PAT&E)  are  included. 

1.5  Assumptions.  In  recommending  T&E  and  IW&T  actions  for  the 
DDN,  we  have  assumed  that: 

(a)  There  is  no  Defense  Systems  Acquisition  Review  Council 
(DSARC)  milestone  process  applicable  to  the  DDN. 


(b)  There  is  no  Lead  Military  Department  (LMD)  to  accomplish 
DDN  Operational  Test  and  Evaluation  (OT&E)  as  defined  in 
DoD  Directive  5000.3. 

(c)  Operating  tests  described  in  this  and  other  documents 
relevant  to  the  DDN  substitute  for  the  operational  tests 
according  to  5000.3 

(d)  The  structure  for  the  DDN  test  program  presented  in  the 
revised  Program  Plan  of  May  1982  and  updated  by  the  Draft 
Management  Engineering  Plan  of  December  1982  provides  the 
basis  for  test  planning. 

(e)  Prioritization  of  test  recommendations  in  groups  1 
through  5,  as  described  in  Paragraph  2.3,  meets  the  DDN 
PMO  requirement  for  rank  ordering  the  recommendations. 

(f)  Test  recommendations  for  the  DDN  can  be  made  on  the 
assumption  that  all  required  resources  will  be  available 
to  accomplish  the  test  program. 

(g)  Operational  control  of  the  DDN  network  by  DCA  during  the 
period  operating  tests  are  conducted  will  be  sufficiently 
strong  to  complete  valid  testing  requirements. 

1.6  Methodology.  Figure  1-2  depicts  the  methodology  used  in 
performing  ‘Subtask  1. 


Figure  1-2.  Study  Methodology 


2.  TEST  AND  EVALUATION  REQUIREMENTS  ANALYSIS 

2.1  DDN  Test  Situations. 

2.1.1  Testing  at  the  Component  Level.  The  purpose  of  component 
testing  is  to  verify  that  each  unit  of  hardware  or  software 
performs  in  accordance  with  its  design  specifications.  It  assures 
that  the  development  of  the  component  has  been  completed 
satisfactorily  and  that  hardware  and  software  components  will  be 
trouble-free  when  integrated  at  a  site.  It  is  conducted  before 
installation  of  hardware  and  software  at  a  site  and  before  their 
integration  into  the  network.  Component  testing  also  includes 
testing  of  lines  as  appropriate  and  required. 

2. 1.1.1  Objectives.  The  following  specific  objectives  can  be 
achieved  through  component  testing: 

(a)  Verify  that  components  have  been  developed  in  accordance 
with,  and  meet  all,  design  specifications. 

(b)  Verify  that  hardware  and  software  components  have  been 
properly  integrated. 

(c)  Obtain  data  on  component  performance  in  either  a 
laboratory  or  operational  facility. 

2. 1.1. 2  Scope.  The  scope  of  testing  is  different  for  hardware 
and  software  component  items.  Hardware  testing  may  be  conducted 
in  any  or  all  of  seven  areas: 

(a)  Design  Verification  -  does  the  item  meet  design  and 
performance  specifications? 

(b)  Reliability  -  will  the . equipment  support  the  user? 

(c)  Environmental  -  does  the  equipment  perform  in  the 
required  operational  environment? 

(d)  Maintainability  Prediction  -  what  is  predicted  downtime 
as  it  relates  to  required  operating  time? 


(e)  Human  Engineering  -  how  can  the  effectiveness  of 
man-machine  interfaces  be  maximized? 

(f)  Electrical  -  EMI,  ESD,  TEMPEST,  and  HEMP  testing 

(g)  Endurance  -  does  the  equipment  meet  its  performance 
requirements  for  its  intended  life? 

Software  tests  may  be  conducted  in  any  of  three  areas: 

(a)  Performance  -  does  the  software  perform  to  its 
specification? 

(b)  IVV&T  -  does  the  software  meet  design  specifications  as 
independently  determined? 

(c)  Quality  Assurance  -  does  the  already  existing  or  acquired 
software  function  properly? 

2. 1.1. 3  Basic  Scenarios.  Government  involvement  in  component 
testing  normally  begins  with  selected  testing  of  a  component  under 
development  in  the  developer's  facility.  The  amount  of  this 
involvement  will  vary  according  to  several  factors.  If  the 
component  being  developed  is  very  complex,  the  Government  may 
direct  or  conduct  testing  at  each  stage  of  development  to  assure 
that  each  has  been  completed  satisfactorily.  Similarly,  if 
developing  the  component  requires  the  development  of  new 
technology,  testing  at  each  stage  of  development  may  be  dictated. 
Another  factor  is  the  cost  of  a  failure  of  the  development 
effort.  When  component  development  costs  are  low,  the  cost  of 
testing  during  development  may  not  be  justified,  but  as 
development  costs  increase  the  benefit  from  developmental  testing 
likewise  increases. 

While  these  factors  must  be  considered  for  both  hardware  and 
software,  they  are  most  likely  to  dictate  Government-directed 
testing  during  the  development  of  software,  since  software  costs 
are  already  high  and  still  increasing  out  of  proportion  to  the 
rate  of  cost  increase  for  hardware.  Experience  has  shown  that  it 
is  best  to  monitor  and  test  software  during  its  development  rather 


software  during  its  development  rather  than  to  allow  errors  to 
remain  undetected  until  the  software  is  put  into  operation.  If 
the  software  development  effort  is  not  successful,  it  is  difficult 
to  recoup  the  resources  expended,  and  remedial  efforts  may  not  be 
possible  within  the  limits  of  schedules  and  available  resources. 

One  strategy  that  can  help  prevent  these  problems  during  the 
development  of  new  software  is  Independent  Verification, 
Validation,  and  Testing,  the  use  of  an  independent  authority  to 
conduct  checks  and  tests  during  the  software  development  process. 
IW&T  increases  the  probability  of  error  detection  and  correction, 
thereby  enabling  successful  completion  of  the  development  program. 

Not  all  DDN  software  will  be  newly  developed.  In  certain  cases 
the  DDN  PMO  will  acquire  software  which  has  been  developed  under 
other  auspices.  In  these  instances,  a  well  developed  and  properly 
executed  quality  assurance  (QA)  program  is  the  appropriate  method 
for  assuring  that  the  software  meets  performance  specifications. 

When  development  of  a  hardware  or  software  component  is  complete. 
Government  acceptance  testing  is  conducted.  This  testing  may  be 
done  in  the  developer's  facility  or  in  a  Government  test 
facility.  This  testing  nqrmally  constitutes  an  end  item  operating 
check  at  varied  levels  of  loading,  stress,  and  environmental 
extremes.  The  nature  of  the  components  governs  the  nature  of  the 
test  facility  required.  Some  components  may  be  tested  very 
simply.  For  other  components,  a  test  network  or  special  test 
driver  could  be  required  to  fully  verify  hardware  and  software 
component  capabilities. 

This  level  of  testing  ensures  that  a  hardware  or  software 
component  is  ready  to  be  installed  at  a  site  and  integrated  into 
the  network.  While  additional  testing  of  components  will  take 
place  in  the  form  of  integration  and  network  testing,  this  level 
of  component  testing  assures  that  site  or  network  integration  can 
begin  with  a  high  degree  of  confidence  in  the  component's 
performance. 


2.1.2  Site  Transition  Testing.  Site  transition  testing  consists 
of  those  tests  needed  to  ensure  that  DDN  installed  equipment  has 
been  properly  integrated,  operates  in  accordance  with  design 
specifications,  and  provides  the  subscriber  with  the  required 
functional  capabilities.  Site  transition  testing  includes  testing 
at  switching  node  sites  to  ensure  proper  operation  of  the  DDN 
backbone  elements  and  testing  at  subscriber  sites  to  bring  user 
host  computers  and  terminals  on-line.  It  is  an  integral  part  of 
site  transition  and  cutover,  and  when  completed  successfully, 
assures  that  the  site  is  interacting  correctly  with  the  DDN. 

2. 1.2.1  Objectives.  The  objectives  of  site  transition  testing 
are: 

(a)  To  assure  that  site  hardware  and  software  are  properly 
interconnected  and  operationally  compatible  with 
connected  DDN  facilities  and  equipment. 

(b)  To  assure  that  the  site  meets  all  protocol  requirements 
for  interface  formats  and  codes. 

(c)  To  assure  that  proper  operational  techniques  have  been 
developed  for  processing  DDN  traffic  under  normal  and 
abnormal  conditions. 

2. 1.2. 2  Scope.  Subscriber  site  transition  testing  is  conducted 
primarily  to  determine  the  readiness  of  a  site  to  accomplish  the 
transition  to  the  DDN  and  to  determine  that  the  transition  has 
been  accomplished  successfully.  Testing  therefore  consists  of: 

(a)  On-site  inspection,  acceptance,  and  integration  of  DDN 
supplied  components 

(b)  Diagnostics  of  hardware  and  software 

(c)  Testing  of  system  functional  capabilities  with  DDN 
equipment  installed. 

2. 1.2. 3  Basic  Scenarios.  Subscriber  site  transition  testing 
should  be  conducted  as  a  part  of  each  subscriber  site's  transition 


to  the  DON.  Subscriber  equipment  testing  is  the  subscriber's 
responsibility;  the  DDN  PMO  is  responsible  for  DDN  supplied 
equipment  and  for  switching  node  sites.  Normally  site  transition 
testing  should  follow  this  sequence: 

(a)  Inventory  of  all  equipment  and  documentation  necessary  to 
accomplish  cutover  for  the  site. 

(b)  Inspection  of  all  equipment  to  ensure  its  satisfactory 
installation  and  readiness  for  testing. 

(c)  Diagnostic  testing  of  equipment  to  ensure  its  readiness 
for  operational  testing. 

(d)  Operational  testing  of  site  equipment  independent  of  the 
network. 

(e)  Operational  testing  of  site  equipment  utilizing  the 
partitioned  network. 

(f)  Operational  testing  of  site  equipment  utilizing  the  full 
network. 

Synergistically,  these  actions  will  allow  a  successful  transition 
to  the  DDN.  Detailed  procedures  for  conducting  the  necessary 
tests  should  be  described  in  the  test  plan  and  accompanying 
documents.  The  test  plan  should  be  written  to  support  the 
transition  and  cutover  plans  for  the  site. 

In  order  to  conduct  site  transition  testing,  close  coordination 
with  operators  and  system  users  is  necessary.  DDN  transition 
testing  should  be  accomplished  with  as  little  network  disruption 
as  possible.  Nevertheless,  some  restriction  of  operational 
traffic  will  be  unavoidable.  This  restriction  can  be  minimized 
through  detailed  planning  and  well  coordinated  transition, 
cutover,  and  test  efforts. 

2.1.3  DDN  Network  Integration  Testing.  Network  integration 
occurs  as  the  DDN  develops  in  levels  through  the  integration  of 
MINET,  the  DoD  Intelligence  Information  System  (DODIIS),  the 
Strategic  Air  Command  Digital  Information  Network  (SACDIN) ,  the 


WWMCCS  Intercomputer  Network  (WIN),  and  the  SECRET  Network  with 
the  MILNET.  Testing  at  each  stage  in  the  development  of  DDN  is 
intended  to  verify  the  capability  of  network  hardware  and  software 
to  fulfill  the  requirements  of  the  network  functional  baseline. 
These  operating  tests  conducted  on  existing  networks  may  be 
considered  in  the  context  of  operational  tests,  according  to  DoD 
Directive  5000.3. 

2. 1.3.1  Objectives.  Network  integration  objectives  are: 

(a)  Exercise  and  test  all  areas  of  system  hardware  and 
software  to  verify  that  they  meet  the  total  requirement 
of  the  specification. 

(b)  Demonstrate  that  the  packet-switching  (PS)  software  and 
hardware  will  provide  packet  integrity  and  protection 
with  reliable  security  provisions  under  all  operating 
conditions  while  simultaneously  meeting  requirements  for 
traffic  flow,  speed  of  service,  routing,  throughput, 
reliability,  and  availability. 

(c)  Assure  that  the  switching  subsystems,  trunking  networks, 

'  and  access  area  subsystems  are  integrated  for  proper 

network  operation.  . 

(d)  Demonstrate  the  capability  of  the  network  control 
facilities  to  monitor  and  control  the  network  under  both 
normal  and  abnormal  conditions. 

(e)  Verify  and  validate  all  applicable  operational  manuals 
and  support  documentation. 

(f)  Validate  system  performance  in  providing  data  transfer 
services  for  various  transaction  scenarios. 

(g)  Assure  that  the  network  meets  security  requirements. 

2. 1.3. 2  Scope.  Network  integration  testing  is  designed  to  verify 
network  performance  and  acceptability  in  the  following  areas: 


(a)  System  performance  -  discrete  testable  technical 
parameters,  such  as  throughput,  defined  in  specifications. 

(b)  Data  handling  and  control  -  functional  and  operational 
software  procedures  and  protocols  that  permit  traffic  and 
services  in  accordance  with  specifications. 

(c)  System  security  -  subnetwork  separation  of  traffic  and 
system  packet  integrity  and  protection. 

(d)  Network  reliability  and  availability  -  the  probability 
that  the  system  will  support  the  user  and  provide  the 
necessary  operating  time. 

2. 1.3. 3  Basic  Scenarios.  Network  integration  testing  should  be 
conducted  in  four  phases  to  provide  a  gradual  transition  onto  the 
network  of  the  equipment  and  procedures  being  tested.  This 
phase-in  will  reduce  to  a  minimum  the  possibility  that 
malfunctions  encountered  during  testing  will  adversely  affect 
operating  portions  of  the  network.  It  will  provide  the  maximum 
assurance  of  proper  operation  before  proceeding  to  the  next  step 
in  integration. 

In  the  first  phase  of  testing,  network  functionality  can  be 
simulated  in  a  test  facility.  The  test  facility  simulates  a 
limited  node  network,  thus  allowing  hardware  and  software 
components  to  be  tested  functionally  at  the  unit,  subsystem,  and 
system  levels;  this  also  allows  their  testing  as  parts  of  a 
simulated  network.  If  this  phase  of  testing  is  completed 
satisfactorily  and  if  it  is  established  that  all  components  are 
functioning  correctly  both  independently  and  as  components  of  a 
test  network,  the  equipment  under  test  can  be  moved  to  an 
operating  site  for  the  next  phase  of  testing. 

The  second  phase  consists  of  intrasite  testing  with  a  host  or 
hosts  to  assure  proper  functioning  of  host/network  interfaces.  It 
also  assures  that  the  network  components  under  test  provide  the 
proper  services  to  host  systems  and  subscribers. 


2-7 


1 


*3 


i 

1 


i 

s 


In  the  third  phase  of  testing,  functionality  is  tested  using  a 
portion  of  the  live  network.  The  purpose  of  partitioning  the 
network  in  this  phase  is  to  conduct  actual  network  testing  while 
limiting  the  risk  of  malfunction  or  disruption  of  the  network.  It 
allows  almost  complete,  low-risk  testing  of  network  functionality 
with  the  minimum  amount  of  interference  to  network  operational 
traffic. 

Phase  four  consists  of  the  remaining  tests  to  be  conducted  with 
the  full  network.  If  previous  phases  have  been  conducted 
correctly,  and  if  problems  have  been  corrected  as  they  appear, 
this  final  phase  of  testing  can  be  carried  out  with  minimal  risk 
to  the  network  and  with  minimal  interruption  of  operational 
traffic.  The  tests  in  this  phase  will  serve  as  a  final  check  for 
correct  network  performance. 

Like  site  transition  testing,  network  integration  testing  must  be 
carefully  planned  and  coordinated.  Test  plans  will  contain 
schedules,  responsibilities,  test  objectives,  descriptions  of  the 
required  test  documentation  and  other  information  required  to 
coordinate  and  conduct  the  necessary  tests. 

2,1,4  Independent  Verification,  Validation,  and  Testing  (IVV&T) . 
IVV&T  is  intended  to  provide  additional  objective  assurance  that  a 
software  component  has  been  developed  to  meet  the  specification 
and  satisfy  the  functional  requirements.  IVV&T  is  most  useful 
when  the  development  process  for  a  component  is  particularly  long 
or  complex  or  when  there  is  a  high  probability  of  undetected 
errors  during  development.  IVV&T  seeks  to  reduce  the  possibility 
of  undetected  errors  during  the  development  process  and  to  provide 
independent  verification  that  the  development  process  has  been 
successful.  IVV&T  should  be  considered  as  developmental  testing 
within  the  purview  of  DoD  Directive  5000.3. 

The  purpose  of  IVV&T  is  to  assure  quality  in  software  under 
development.  The  DDN  PMO  will,  however,  acquire  certain  software 
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that  has  been  developed  by  others  or  assume  responsibility  for 
certifying  software  used  by  others.  IW&T  and  acceptance  testing 
procedures  are  not  appropriate  in  these  cases.  Instead,  strict  QA 
procedures  should  be  used  to  certify  this  software.  QA  procedures 
should  be  developed  on  the  basis  of  approved  QA  policies.  They 
can  fulfill  the  same  purpose  for  acquired  software  as  T&E  can  for 
newly  developed  software. 

2. 1.4.1  Objectives.  As  the  name  suggests,  IW&T  has  three 
objectives.  The  first  is  independent  verification  of  system 
requirements  to  assure  that  they  are  correctly  stated  and  that  all 
requirements  will  be  satisfied  in  the  design. 

The  second  objective  is  to  validate  that  the  software  has  been 
designed  and  developed  in  such  a  way  that  all  valid  requirements 
are  satisfied. 

The  third  objective  of  IW&T  is  to  conduct  tests  necessary  to 
provide  assurance  that  the  software  performs  to  specification. 

2. 1.4. 2  Scope.  IW&T  could  be  appropriately  applied  to  any 
software  system  development  effort.  Its  scope  is  therefore 
unlimited.  Its  most  common  application,  however,  is  in  complex 
software  development  efforts  susceptible  to  undetected  errors 
which  may  jeopardize  the  success  of  the  whole  project.  In  such 
instances,  the  cost  of  IW&T  may  be  small  in  relation  to  the  loss 
incurred  if  the  development  effort  fails  and  requires  major 
post-test  reworking. 

2.2  Identification  of  DDN  Test  Items. 

2.2.1  Testable  Components.  The  DDN  can  be  functionally 
subdivided  into  six  groups:  the  backbone,  access  approaches, 
security,  monitoring  systems,  test  systems,  and  support  systems. 
Each  of  these  functional  groups  can  be  further  broken  down  into 
system  elements  or  components  as  shown  in  Table  2-1.  The  backbone 
components  include  the  C/30  packet  switches  and  the  communication 
trunk  lines.  The  access  approaches  include  the  three  interface 
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Table  2-1.  DDN  Testable  Components 


FUNCTIONAL  AREA 
DDN  Backbone 

DDN/Access  Systems 


DDN/Securi ty 

DDN/Monitor 


DDN/Test  Systems 

DDN/Support  Systems 


ELEMENT  NOMENCLATURE* 

C/30  Switch  Node 
Trunk  Lines 
NAC 

-  Mini-TAC 

-  HFEP 

-  TEP 
HID 

Access-Line  Modems 
TAC 

Statistical  Multiplexers 
IPLI 

Switch  Level  Gates 
Mailbridges 

Network  Monitoring  Center 
Equipment 

Power  and  Environment  Monitor 
Automatic  Line  Restoral  Option 
System  Test  Facility 
Patch  and  Test  Modules 
Mobile  Reconstitution  Van 
Software  Development  Facilities 
Network  Information  Center  (NIC) 


♦Includes  software  as  well  as  hardware  subelements 


methods  by  which  a  host  can  connect  to  the  DDN:  the  Host 
Front-End  Processors  (HFEPs) ,  the  Host  Interface  Devices  (HIDs) , 
and  the  Terminal  Emulation  Processors  (TEPs).  Also  included  in 
the  access  approaches  are  the  two  means  by  which  terminals  may 
access  the  DDN:  the  TACs  and  the  mini-Terminal  Access  Controllers 
(mini-TACs).  The  mini-TACs,  HFEPs,  and  TEPs  can  be  configured 
from  a  common-based  microprocessor  called  the  Network  Access 
Component  (NAC) .  The  statistical  multiplexers  that  will  support 
several  terminals  connected  to  a  remote  mini-TAC  are  also  part  of 
the  access  approaches,  along  with  the  access  line  modems.  The  DDN 
security  elements  are  the  IPLI  and  the  switch-level  gates  which 
will  maintain  community  of  interest  separation  on  DDN,  and  the 
mailbridges.  The  monitoring  system  components  are  the  DDN  MC 
equipment,  the  Power  and  Environment  Monitor  (PEM) ,  and  the 
Automatic  Line  Restoral  Option  (ALRO) .  The  DDN.  Test  systems 
including  a  system  test  facility,  and  limited  digital  patch  and 
test  modules  at  each  subscriber  site.  Finally,  the  DDN  support 
system  consists  of  the  Mobile  Reconstitution  Vans,  software 
development  facilities,  and  the  Network  Information  Center  (NIC) . 

The  purpose  of  the  remainder  of  this  section  is  to  recommend 
general  test  approaches  for  these  components.  Based  on  the 
projected  completion  date  of  their  development,  the  testable 
components  are  categorized  within  the  stages  of  the  planned 
evolution  of  the  DDN.  Test  approaches  cover  software  as  well  as 
hardware.  Testing  of  these  components  may  be  considered  as 
including  developmental  testing  as  well  as  production  acceptance 
testing  within  the  purview  of  DoD  Directive  5000.3. 

2. 2. 1.1  Stages  I  and  II.  Stage  I  of  the  DDN  began  in  fiscal  year 
(FY)  1982  and  extends  through  the  third  quarter  of  FY  1984.  Stage 
II  of  the  DDN  immediately  follows  the  end  of  Stage  I  and  extends 
to  the  end  of  FY  1985.  During  this  stage,  the  NAC  will  be 
available  for  host  access  to  the  DDN. 


With  the  exception  of  the  production  IPLIs,  Blacker  technology 
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security  devices,  and  the  integration  of  elements  in  the  Mobile 
Reconstitution  Vans,  all  system  components  should  be  completed 
within  Stages  I  and  II. 

2. 2. 1.1.1  Hardware.  Hardware  testing  that  applies  to  the  DDN 
elements  falls  into  seven  categories:  design  verification, 
reliability,  environmental,  maintainability,  human  engineering, 
electrical,  and  endurance  testing. 

Design  verification  testing  determines  equipment  compliance  with 
all  applicable  design  and  performance  specifications.  It  is 
usually  applied  to  newly  developed  or  modified  equipment. 

Reliability  testing  provides  the  Government  reasonable  assurance 
that  established  minimum  acceptable  reliability  requirements  have 
been  met  before  items  are  approved  for  mass  production.  It 
applies  to  items  that  are  newly  designed,  have  undergone  major 
modification,  or  have  failed  their  allocated  reliability 
requirements  under  new  system  conditions  (severe  environmental 
stress,  for  example) . 

Environmental  tests  determine  component  performance  under  the 
stress  of  natural  and  induced  environments  peculiar  to  military 
operations.  In  those  cases  where  the  DDN  equipment  is  located  in 
a  controlled  environment  within  a  building,  little  or  no 
environmental  testing  will  be  required.  Nominal  electronic  and 
thermal  stress  tests  are  also  appropriate  for  this  type  of  fixed 
ground  equipment.  Since  environmental  tests  focus  on  basic 
survivability  characteristics  of  equipment,  they  should  be 
performed  concurrently  with  the  early  stages  of  reliability 
testing  for  conditions  expected  to  be  encountered. 

Maintainability  testing  attempts  to  predict  the  number  of  hours 
that  a  system  will  be  inoperative  while  it  is  undergoing 
maintenance.  It  highlights  those  pieces  of  equipments  which. 
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because  of  poor  maintainability,  require  improvement, 
modification,  or  change  of  design.  In  addition,  it  allows  the 
user  to  make  an  early  assessment  of  whether  the  predicted  downtime 
along  with  the  quality  and  quantity  of  personnel,  tools,  and  test 
equipment  are  adequate  and  consistent  with  system  operational 
requirements.  In  order  to  predict  maintainability,  the  tester 
needs  such  information  as  the  failure  rate  of  the  component,  the 
number  of  spares  carried,  the  number  of  test  points,  the  nature  of 
the  test  equipment  to  be  applied,  and  the  manning  schedules  of 
maintenance  personnel.  Provided  that  this  information  does  exist, 
maintainability  testing  can  be  applied  at  any  time  after  the 
development  concept  has  been  established. 

Human  engineering  involves  an  analysis  of  the  relationship  between 
a  piece  of  equipment  and  its  human  interface.  Its  purpose  is  to 
maximize  effectiveness,  simplicity,  efficiency,  reliability,  and 
safety  of  operation,  training,  and  maintenance.  It  should  be  done 
during  the  early  stages  of  equipment  development. 

For  the  purposes  of  DDN ,  electrical  tests  will  consist  of 
electromagnetic  interference  (EMI)  tests,  electrostatic  discharge 
(ESD)  testing,  TEMPEST  testing,  and  high-altitude  electromagnetic 
pulse  (HEMP)  testing.  EMI  testing,  establishes  techniques  for 
measuring  and  determining  the  EMI  characteristics  (emission  of  and 
susceptibility  to)  of  electronic  equipment.  Its  purpose  involves 
identification  and  protection  of  equipment  whose  performance  might 
be  adversely  affected  by  electromagnetic  impulses.  In  addition, 
EMI  testing  identifies  equipment  that  emits  electromagnetic  waves, 
and  methods  of  shielding  it  in  cases  where  the  effects  of  those 
waves  might  prove  to  be  detrimental.  ESD  testing  involves  the 
identification  and  protection  of  equipment  whose  functions  might 
be  adversely  affected  by  static  electricity.  It  is  usually 
applied  to  equipment  which  contains  such  exposed  electronic  parts 
as  microelectronic  and  semiconductor  devices,  thick  and  thin  film 
resistors,  and  chips. 


TEMPEST  testing  is  directed  at  ensuring  that  equipment  does  not 
produce  compromising  emanations.  HEMP  testing  focuses  on 
equipment  protection  in  the  form  of  electromagnetic  shielding, 
line  isolation,  and  surge  arrestment.  These  forms  of  electrical 
testing  are  usually  conducted  during  the  latter  part  of  the  test 
cycle. 

Endurance  testing  is  one  of  the  last  tests  to  be  performed  in  the 
development  cycle.  It  can  be  broken  down  into  three  levels.  The 
first  level  attempts  to  test  an  item  under  normal  conditions  to 
ascertain  that  it  meets  all  of  its  functional  requirements.  The 
second  level  is  overload  testing,  which  stresses  the  component  to 
its  capacity  and  beyond  to  determine:  1)  whether  it  can  withstand 
certain  stress  thresholds  and  still  function;  and  2)  if  it  does 
degrade  under  stress,  how  badly  it  will  degrade,  and  what 
consequences  that  degradation  will  have  on  the  overall  host  system 
functions.  The  third  level  may  repeat  some  or  all  of  the  previous 
tests,  but  will  do  so  in  an  environment  that  closely  approximates 
the  conditions  expected  in  the  operating  environment.  Since 
endurance  testing  goes  beyond  basic  survivability  analysis  and 
stresses  the  test  item  beyond  its  capacity,  it  wi  be  apr-lied  to 
those  items  where  testing  is  considered  most  critical. 

All  system  element  hardware,  excluding  the  production  IPLIs, 
Blacker  technology  security  devices,  and  the  Mobile  Reconstitution 
Vans,  can  be  tested  in  the  first  two  stages  of  DDN 
implementation.  In  analyzing  the  criteria  for  which  tests  to 
apply  to  which  equipment,  the  development  status  of  the  equipment 
appears  to  be  the  most  critical  variable,  along  with  its 
criticality  to  overall  system  performance.  If  a  particular 
component  already  exists,  such  as  an  access-line  modem  or  a  TAC , 
it  should  have  already  proven  itself  in  the  field.  It  therefore 
should  be  subject  to  only  minimal  reliability  and  environmental 
testing  to  verify  and  correct  any  problems  detected  in  operational 


use 


.  If  an  item  is  still  being  developed,  then  it  is  appropriate 
to  test  that  component  more  thoroughly  before  placing  it  in  the 
operational  system.  In  this  instance,  the  more  rigorous  endurance 
tests  should  be  added  to  the  basic  design  verification, 
reliability  and  environmental  tests,  along  with  electrical, 
maintainability,  and  human  engineering  testing.  Table  2-2  gives  a 
complete  breakdown  of  each  DDN  system  element,  its  functional 
area,  the  primary  action  office  responsible  for  it,  its  major 
hardware  subelements,  development  completion  dates,  and  hardware 
test  approaches. 

Because  of  their  unique  functions,  the  trunk  lines  and  the  test 
equipment  associated  with  the  patch  and  test  modules  do  not  adhere 
to  the  standard  type  of  component  testing.  For  trunk  lines,  tests 
which  address  the  problem  of  bit  error  rate  in  data  transmission 
should  be  applied.  One  function  of  the  MC  is  continual  testing  of 
operational  circuits  with  in-use  software  to  determine  error 
rates.  For  the  test  equipment  associated  with  the  test  modules, 
calibration  tests  should  be  performed. 

Hardware  component  testing  should  be  done  at  the  vendor's  facility 
when  approved  by  the  Government.  If  it  is  determined  that  a 
different  test  location  is  necessary  (endurance,  TEMPEST,  and  HEMP 
testing  would  fall  into  this  category) ,  a  Government-approved  test 
facility  should  be  used.  In  any  case,  the  Government  should 
reserve  the  right  to  witness  any  or  all  testing. 

2. 2. 1.1. 2  Software.  There  are  three  basic  approaches  to  testing 
software  components.  The  first  is  performance  testing  at  the  time 
of  acceptance  from  the  developer,  the  second  is  utilization  of 
iVv&T,  and  the  third  is  the  implementation  of  a  well  defined  QA 
program. 

Performance  testing  is  usually  conducted  at  the  time  of  acceptance 
by  a  test  agency  independent  of  the  procuring  agency.  Performance 
testing  is  intended  to  exercise  all  the  functions  of  the  software 
to  demonstrate  that  it  is  reasonably  error-free.  This  testing 
ensures  that: 
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ENVIRONMENTAL 
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MEDIUM  SPEEO 

ENOURANCE 
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1.  HARDWIRED  LOGIC 
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1.  TERMINAL  ACCESS 
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FMO 

COMPUTER 

(STAGE  10 

OESIGN 

PROTOCOLS 

2.  MICROCOMPUTER 

FY  IB 

VERIFICATION/ 

2.  HOST-HOST  SOFTWARE 

TBO 

IWBT 

CONTROLLED 

(STAGE  10 

RELIABILITY/ 

(TCP.  IP) 

J.  most/uni  sus 

FY  M 

ENVIRONMENTAL 

3  HOST  IMP  SOFTWARE 

TBO 

INTERFACE 

(STAGE  Ml 

MAINTAINABILITY/ 

4.  CMOS  CONTROL 

FY  88 

4  UNISUa/Q-SUS 

FY  IB 

electrical/ 

PROGRAM 

(STAGE  ID 

AO  AFTER 

(STAGE  II) 

ENOURANCE 

S.OSUVHDH 

FY  BE 

validation 

INTERFACE 

(STAGE  (0 

ACCESS- LINE 

OON 

1  ASYNCHRONOUS 

NONE 

MODEMS 

FMO 

(0-300. 0-1200  BAUD) 
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ENVIRONMENTAL 

(2400. 4800. 9S00  BAUD- 

VALIDATION 

RACK -MOUNTED) 

TP  No.  063-1061 2-C-1 


Table  2-2.  DDN  Developmental  Testing 
System  Element/Subelement  Level 
(Page  2  of  2) 


FUNCTIONAL  AREA 

KLIMS  NT 
WOSUNCLATUm 

RESPONSIBILITY 

HAROWARS  SUM  LIME  NTS 

DEVELOPMENT 
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ENVIRONMENTAL 

VALIDATION 

1  TERMINAL  ACCESS 
PROTOCOLS 

2  HOST -MOST  SOFTWARE 
(TCP.  IP) 

3.  HOST  IMP  SOFTWARE 

4.  CHARGEBACK 

algorithm 

AVAILABLE 

AVAILABLE 

AVAILABLE 

QUALITY 

ASSURANCE 

STATISTICAL 

MULTIPLEXERS 

DON 

FMO 

t  TIMCPLEX  MOOEL 

IM40V  MB03.  MBCMI 

available 

RELIABILITY/ 

ENVIRONMENTAL 

VALIDATION 

NONE 

OOH/SECURITY 

INTERNET 

PRIVATE 

LINE  INTERFACE 

IIPLII 

OON 

FMO 

1 .  KC  §4  CRYPTO 

2  MC  88000  RACKET 
FROCESSORS 

fy  86 

(STAGE  II) 
FY86 

(STAGE  Mil 

DESIGN 

VERIFICATION/ 
RELIABILITY/ 
ENVIRONMENTAL/ 
MAINTAINABILITY/ 
ELECTRICAL/ 
ENOURANCE  VALID 

1  IMP  IMP  SOFTWARE 

2.  HOST  IMP  SOFTWARE 

FY  BB 
(STAGE  ID 

FY  85 
(STAGE  ID 

IV  VST 

SNITCH  LEVEL 
GATES 

DON 

PVO 

1  SWITCHGATE 

HARDWARE 

pm 

PERFORMANCE 

VALIDATION 

MAIL  BRIDGE 

DON 

FMO 

1  MAIL  BRIDGE 

HARDWARE 

TBD 

1  MAIL  BRIDGE 
SOFTWARE 

TBD 

PERFORMANCE 

VALIDATION 

DON/MONITOR 

DON 

FMO 

C/70  MINICOMPUTER 

a.  EXECUTION  MACHINE 

b.  DISKS 

c.  ONE  S00  bp.  TAPE 

d.  TERMINALS 

t.  PRINTERS 

FY  84 

(STAGE  0 

RELIABILITY/ 

ENVIRONMENTAL 

VALIDATION 

1.  UNIX  OPERATING 
SYSTEM 

2.  HOST  IMP  PROTOCOL 

3.  NU  SOFTWARE 

4  CHARGEBACK 

algorithm 

AVAILABLE 

FY  BB 
(STAGE  HI 

B6  JANUARY 

(STAGE  III 

QUALITY 

ASSURANCE 

POWER  AND 
ENVIRONMENT 
MONITOR 

OON 

FMO 

1.  SENSORS 

2.  CONTROLS 

H 

RELIABILITY/ 

ENVIRONMENTAL 

VALIDATION 

1 .  HOST  IMF  PROGRAM 

2.  DATA  COLLECTION 

AND  CONTROL 
PROGRAM 

88  JANUARY 
(STAGE  II) 

BB  JANUARY 
(STAGE  II) 

PERFORMANCE 

VALIDATION 

AUTOMATIC  LINE 
RESTOAAL 
OPTION 

DON 

FMO 

1  MODEM 

2.  AUTOMATIC  CALLING 
UNIT 

3.  DATA  At  SWITCH 

4.  DATA  LOOPBACK  SWITCH 

5.  LINE  LOOPSACK  SWITCH 

S.  MASTER  CONTROLLER 

UNIT 

7  SLAVE  CONTROLLER 

UNIT 

8.  DIAL  BACKUP  ANSWER 

OPTIONAL 

RELIABILITY/ 

ENVIRONMENTAL 

VALIDATION 

1  CONTROL  PROGRAM 

OPTIONAL 

PERFORMANCE/ 

validation 

don/tsst 

SYSTEMS 

TEST  FACILITY 

OON 

FMO 

M 

REFER  TO 
INDIVIDUAL 

elements 

(ACQUISITION 

oate 

88  MARCH) 

REFER  TO  INDIVIDUAL 
ELEMENTS 

■ 

OON 

FMO 

1.  VP  PATCH  ANO  TEST 

EQUIPMENT 

2.  DIGITAL  PATCH  ANO 

TEST  EQUIP 

3.  DIOITAL  SWITCHING 
MOOULC 

4.  TUT  EQUIP 

a.  ANALOG/OIGITAL 

TEST  FANIL 

b.  CIA  SRSAKOUT 

TEST  PANEL 

e.  OATA  LINK 

ANALYZER 

AVAILABLE 

AVAILABLE 

AVAILABLE 

AVAILABLE 

ENVIRONMENTAL/ 

CALIBRATION 

VALIDATION 

OATA  LINK  ANALYZER 

software 

PERFORMANCE 

validation 

DON/1UPPORT 

SYSTEMS 

MOEILI 

RSCONSmUTtON 

VAN 

OON 

FMO 

1.  C/30i 

2.  CTO 

3.  MINl-TAC 

4  IPU 

B.  MOOEMS 

S.  PATCH  B  TEST  EQUIP 

REFER  TO 

INDIVIDUAL 

elements 

REFER  TO  INDIVIDUAL 
ELEMENTS 

NETWORK 

INFORMATION 

CENTER 

INICI 

OON 

FMO 

NIC 

HARDWARE 

AVAILABLE 

reliability/ 

ENVIRONMENTAL 

VALIDATION 

NIC 

SOFTWARE 

AVAILABLE 

QUALITY 

assurance 
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(a)  The  software  can  satisfy  all  applicable  system  and 
program  performance  requirements. 

(b)  The  software  can  properly  interface  with  all  interacting 
systems  specified  in  the  program  performance  requirements 

(c)  The  software  conforms  to  all  applicable  requirements  for 
design  and  documentation. 

The  usual  criteria  for  ascertaining  program  conformance  to 
specifications  are  appropriate  functional  response,  accuracy  (low 
error  rate)  and  timeliness  (quick  response  time)  . 

Testing  the  software  under  stress  conditions  is  an  inherent  part 
of  performance  testing.  It  is  usually  conducted  by  operating  the 
program  under  test  at  saturation  levels  for  certain  periods  during 
the  performance  testing.  The  purpose  is  to  demonstrate  that 
program  degradation  will  not  be  catastrophic  if  the  program  is 
stressed  at  levels  slightly  past  its  design  limits. 

In  certain  cases  IVV&T  of  software  may  be  directed,  in  addition  to 
the  performance  testing  that  should  be  conducted  at  acceptance.  A 
decision  to  bear  the  additional  cost  of  IVV&T  will  usually  be  made 
when  the  software  to  be  developed  is  particularly  critical  to 
satisfactory  network  operation  or  when  its  development  is  expected 
to  be  unusually  difficult  or  complex.  In  these  cases  it  may  be 
cost-effective  for  the  PMO  to  engage  an  independent  contractor  to 
monitor  the  software  development  effort.  Since  the  IW&T  is 
conducted  during  the  period  of  development,  the  chances  that  the 
software  will  prove  unsatisfactory  during  acceptance  testing  are 
significantly  reduced.  The  method  used  in  IVV&T  is  to  verify 
requirements  and  validate  that  the  software  fulfills  them  at  each 
stage  of  its  development.  This  approach  reduces  the  possibility 
that  errors  during  development  will  accumulate  and  remain 
undetected  until  final  testing.  Since  the  cost  of  IVV&T  is 
usually  significant,  the  decision  to  use  it  should  be  made  by 
assessing  the  impact  of  a  failure  to  successfully  complete  a 
critical  software  module  on  time  against  the  cost  of  IW&T  during 
its  development. 


A  third  approach  to  software  testing  is  to  employ  QA  policies  and 
procedures  to  assure  initial  and  continued  satisfactory 
performance.  This  approach  will  be  most  appropriate  for  the  DDN 
PMO  in  the  case  of  acquired  or  assumed  software.  In  the  case  of 
software  developed  elsewhere  but  acquired  by  the  DDN  PMO,  QA 
audits  should  be  conducted  to  determine  the  condition  of  the 
software  and  its  documentation  when  acquired.  Rigorously 
applying  QA  procedures  will  be  the  only  method  available  to  the 
PMO  to  assure  that  acquired  software  meets  the  same  standards  as 
software  developed  under  the  direction  of  the  PMO. 

When  the  DDN  PMO  assumes  responsibility  for  certifying  software 
developed  and  maintained  elsewhere,  rigorous  QA  will  also  be 
necessary.  Requiring  appropriate  audits,  adequate  maintenance  of 
documentation,  and  other  QA  tools  will  be  the  only  means  available 
to  the  PMO  for  ensuring  that  such  software  is  maintained  properly 
and  continues  to  function  satisfactorily. 

Regardless  of  the  testing  approach  used,  all  software,  excluding 
that  used  in  the  production  IPLIs,  Blacker  technology  security 
devices,  and  the  Mobile  Reconstitution  Vans,  can  be  tested  in  the 
first  two  stages  of  DDN  development. 

Analysis  of  the  criteria  used  for  developing  software  testing 
recommendations  shows  that  for  software,  as  well  as  for  hardware, 
the  most  important  considerations  are  development  status  and 
criticality  to  the  system. 

If  these  criteria  are  applied  to  the  DDN  software,  certain 
conclusions  can  be  made.  Because  the  software  for  the  IPLIs  and 
the  network  access  approaches  is  critical  to  the  performance  of 
the  DDN,  and  since  much  of  it  is  still  being  developed,  the  more 
extensive  IVV&T  procedures  should  be  undertaken.  In  cases  where 
much  of  the  software  already  exists,  as  in  the  case  of  the  C/30 
IMP  and  TAC,  a  comprehensive  QA  test  program  should  be  used. 

Also,  as  a  minimum,  extensive  QA  should  be  performed  on  additional 


C/30  IMP  and  TAC  and  C/70  MC  software  such  as  the  DDN  chargeback 
algorithm.  Finally,  in  those  system  elements  where  the  software 
is  still  under  development  but  does  not  directly  impact  network 
performance,  as  in  the  case  of  the  PEM  or  system  test  facility, 
the  performance  test  approach  should  be  used. 

Table  2-2  gives  a  complete  breakdown  of  system  element  software 
with  expected  completion  dates  and  test  approach  recommendations. 

2. 2. 1.2  Stage  III.  Stage  III  of  the  planned  evolution  of  the  DDN 
will  extend  through  FY  1986.  In  this  stage,  the  number  of  IPLIs 
will  be  sufficiently  high  so  that  the  three  subnetworks  can  be 
combined  into  a  single  network  supporting  multiple  levels  of 
security. 

2. 2. 1.2.1  Hardware.  The  only  system  elements  involved  in  this 
time  frame  are  the  production  IPLIs  and  the  Reconstitution  Vans. 
The  hardware  testing  approach  delineated  in  Paragraph  2. 2. 1.1.1 
should  also  apply  to  these  elements.  Table  2-2  lists  the 
approaches  to  be  taken  for  these  two  items. 

2. 2. 1.2. 2  Software.  Software  testing  will  follow  the  rationale 
described  in  Paragraph  2. 2. 1.1. 2. 

2. 2. 1.3  Stage  IV.  Stage  IV  is  the  final  stage  of  the  DDN 
evolution.  During  this  stage,  any  additional  subscribers  that 
need  to  access  the  network  will  be  brought  on-line  by  the  DDN  PMO. 

2. 2. 1.3.1  Hardware.  The  final  system  element  of  the  DDN,  the 
Blacker  security  device,  should  be  completed  during  this  stage. 
More  data  is  required  on  this  device  before  testing  approaches  can 
be  determined. 

2. 2. 1.3. 2  Software.  The  software  subelements  of  the  Blacker 
security  device  scheduled  for  completion  in  FY  1987  have  yet  to  be 
determined. 


2.2.2  Site  Transition  Testing 


(a)  Site  transition  testing  includes  two  basic  test 

situations:  (1)  testing  to  insure  that  the  node  site 

equipment  is  installed  and  functioning  properly,  and  (2) 
testing  required  to  enable  a  subscriber  to  come  on-line 
as  a  DDN  participant.  A  testing  approach  that  minimizes 
interruption  of  ongoing  site  operations  should  be  used. 
Subscribers  will  enter  DDN  through  a  host  port,  TAC, 
mini-TAC,  HFEP,  or  TEP  and  should  ensure  that  their  own 
site  equipment  is  working  to  an  established  baseline. 

The  host  computer  should  be  interfaced  off-line  to  assure 
interoperability  without  degradation  to  the  net.  Once 
this  is  accomplished,  controlled  operating  tests  should 
be  conducted  in  a  NAC  wraparound  or  partitioned  net 
situation.  The  scope  of  testing  is  limited  to 
operational  suitability  and  system  functional 
capability.  The  MC  provides  a  loop-back  capability  for 
checking  subscriber  software  efficacy.  Subscriber  site 
testing  should  also  include  certifying  vendor-software 
interface. 


(b)  The  testing  concept  for  site  transition  testing  starts 
with  assessment  and  analysis  of  existing  performance  and 
proceeds  with  testing  of  DDN  hardware  and  software 
required  for  connectivity.  Hardware/software  integration 
testing  of  site  components  and  backbone  components  is 
performed  to  assure  readiness  for  network  integration. 


(c)  Assumptions  and  conditions.  The  assumptions  and 
conditions  for  hardware,  software,  test  tools,  and 
special  test  conditions  are  as  follows: 


(1)  Pass/Fail  criteria  have  been  objectively  stated. 

(2)  The  sites  will  be  capable  of  supporting  a  test 
environment  with  their  existing  hardware  inventory. 


(3)  The  software  components  will  support  full  network 
operations  in  consonance  with  the  security 
environment . 

(4)  All  local  host  and  related  software  elements  are 
compatible  and  fully  integrated. 

(5)  System  monitoring  and  data  generation/ 

reduction  packages  will  be  made  available  to  all  who 
are  involved  in  testing  or  problem  identification. 

(6)  Checkout  of  participants,  file  space,  and  systems 
files  will  occur  before  scheduled  testing  begins. 
Operating  tests  will  be  conducted  using  the  live 
network;  however,  sites  should  be  prepared  to  fall 
back  to  an  operational  system.  Sites  also  should 
ensure  that  operational  messages  are  not  processed 
when  sites  are  running  under  test  conditions. 

(7)  DDN  testing  will  not  extend  into  the  subscriber's 
security  system  or  procedures  as  he  practices  them  on 
his  host. 

(d)  Scheduling.  Site  testing  should  commence  after  the  site 
representative  and  the  DDN  PMO  representative  have 
verified  site  topology  and  complete  site  preparation. 
Testing  times  should  be  coordinated  with  the  DDN  Test 
Director. 

(e)  Testing.  When  completed,  site  transition  testing  should 
verify  objectives.  The  program  will  include  testing  of 
both  hardware  and  software,  evaluation  of  subsystem 
integration  and  total  system  operation,  and  demonstration 
of  Test  System  subscriber  interfaces.  To  be  most 
effective  and  reduce  the  probability  of  duplicating 
tests,  this  program  will  be  coordinated  with  network 
integration  and  cutover  and  implementation  efforts. 
Government  acceptance  of  the  system  will  be  based  on 


successful  testing  or  demonstration  of  all  end  items 
through  this  integrated  system  test  approach.  Individual 
site  tests  should  be  conducted  for  host  and  subnet  and 
DDN  subscriber  interfaces.  Using  contractor-  developed 
plans  and  procedures,  site  testing  will  be  performed  in 
three  phases:  a  pre-shakedown  phase  involving 
installation  and  checkout  of  individual  configuration 
items  (including  software)  and  subsystems,  a  shakedown 
phase  (dry  run)  to  confirm  interoperability  of  the  site 
subsystems  and  to  ensure  readiness  for  the  final  phase, 
the  formal  site  operational  (acceptance)  test,  which  will 
be  performed  under  Government  direction. 

(f)  Site  Test  Objectives.  The  following  objectives  apply 
broadly  to  all  formal  site  acceptance  tests  to  be 
performed: 

(1)  Installation  and  facility  performance  meets  all 
requirements  of  the  DDN  functional  baseline. 

(2)  The  site  hardware  and  software  are  properly 
interconnected  and  operationally  compatible  with  DDN 
facilities  and  equipment. 

(3)  The  site  meets  all  protocol  requirements  for 
interface  formats  and  codes. 

(4)  Proper  operational  techniques  have  been  developed  for 
processing  traffic  under  normal  and  abnormal 
conditions. 

(5)  Site  testing  is  conducted  to  determine  operational 
suitability — that  is,  can  the  user  get  his  job  done 
with  DDN  under  operational  conditions.  In  addition, 
testing  of  subscriber  interface  hardware  and  software 
will  include  full  operational  compatibility  and 
security  testing  with  the  host  computers  and 
terminals  designated  as  test  support  systems  or 
components. 


It  is  essential  that  problems  identified  during  testing  be  noted 
and  documented,  and  provision  made  for  analysis  and  possible 
resolution  and  correction.  Hardware  and  software  fixes  should  be 
submitted  to  the  DDN  PMO  for  a  determination  of  the  amount  of 
rework  and  retesting  required  before  field  use. 


2. 2. 2.1  Stages  I  and  II  Site  Transition  Testing. 


Hardware 


Software 


C/30  Switch  Node 


Trunk  Lines 


1.  Precedence/Preemption 

2.  Logical  Addressing 

3.  User  Authentication 

4.  X.25  Interface 

5.  Chargeback  algorithm 

None 


NAC 

Mini-TAC 


HFEP 


1.  Terminal  Access  Protocols 

2.  Host-Host  Software 
(TCP/IP) 

3.  Host-IMP  Software  (HDH , 
HDLC) 

4.  ASYNC  Terminal  Software 

5.  SYNC  Terminal  Software 

1.  Terminal  Access  Protocols 

2.  Host-Host  Software 
(TCP/IP) 

3.  Host-IMP  Software  (HDH, 
HDLC) 

4.  CMOS  Control  Program 


TEP 


1.  Communications  Software 

2.  Terminal  Emulator 
Software 
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3.  Host-Host  Software 
(TCP/IP) 


Hardware 


Software 


HID 

Access-line  Modems 
TAC 

Statistical  Multiplexers 
Switch  Level  Gates 
Mailbr idges 


4.  Host-IMP  Software  (HDH, 
HDLC) 

5.  ASYNC  Terminal  Software 

6.  SYNC  Terminal  Software 

1.  Terminal  Access  Protocols 

2.  Host-Host  Software 
(TCP/IP) 

3.  Host-IMP  Software 

4.  CMOS  Control  Program 

None 

1.  Terminal  Access  Protocols 

2.  Host-Host  Software 
(TCP/IP) 

3.  Host-IMP  Software 

4.  Chargeback  Algorithm 

None 

1.  Switchgate  Software 
1.  Mailbridge  Software 


* 

2. 2. 2. 2  Stage  III  Site  Transition  Testing.  This  stage  assumes 
all  sites  are  on-line.  If  a  site  is  not  on-line,  repeat  Stages  I 
and  II  testing  plus  Stage  III  testing. 

Hardware  Software 


IPLI 


.  IMP-IMP  Software 
2.  Host-IMP  Software 


2. 2. 2. 3  Stage  IV  Site  Transition  Testing.  This  stage  assumes  all 
sites  are  on-line.  If  a  site  is  not  on-line,  repeat  Stage  I  and 
II  plus  Stage  IV  testing. 

Hardware  Software 

Blacker  TBD 

equipment 

2.2.3  DON  Subsystem  Integration  Testing.  This  paragraph 
addresses  the  sequencing,  methodology,  and  types  of  DDN 
integration  testing  required  to  establish  an  initial  operating 
capability  (IOC)  for  DDN.  This  integration  testing  must  make 
certain  that  the  combination  of  all  the  individual  subnetworks 
retains  the  original  technical  performance  and  subscriber  services 
of  each.  It  must  also  ensure  that  required  subnetwork  separation 
and  security  of  data  are  maintained  and  demonstrated.  Testing  at 
this  level  assumes  that  successful  component  and  subscriber  site 
transition  testing  have  been  accomplished. 

Integration  testing  of  the  DDN  will  consist  of  five  progressive 
phases.  These  phases  are  conducted  in  the  following  order: 

(a)  ARPANET  split  forming  MILNET  and  Experimental  ARPANET;  and 
MINET  formation  and  integration  into  MILNET 

2  2 

(b)  C  (WIN)  and  DoDIIS  integration  to  form  C  I 

2  2 

(c)  C  I  and  SACDIN  integration  to  augment  C  I 

2 

(d)  C  I  and  SECRET  Network  integration 

2 

(e)  C  I/SECRET  Network  and  MILNET  integration  to  establish 
a  DDN  IOC. 

The  objective  of  DDN  integration  testing  is  to  verify  the 
capability  of  the  network  hardware  and  software  to  fulfill  network 
requirements  as  specified  in  the  baseline  documents.  Four 
functional  categories  of  integration  testing  have  been  identified, 
together  with  their  corresponding  test  objectives.  These  are: 


inu 
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(a)  System  performance  parameter  tests  -  To  verify  the 
ability  of  the  system  to  satisfy  user  and  host 
requirements  at  a  specified  level  of  network  performance. 

(b)  Network  data  handling  and  control  tests  -  To  demonstrate 
that  the  system's  functional  and  operational  structure, 
along  with  designed  procedures  and  protocols,  will 
actually  transfer  messages,  transactions,  and  data  from 
one  subscriber  to  another  through  the  network.  In 
particular,  to  demonstrate  that  the  system  provides 
required  interactive,  query-response,  narrative,  and  bulk 
data  transfer  services.  Also  to  verify  the  adequacy  of 
traffic  management  functions  for  network  control,  the 
system's  capability  to  carry  out  the  initialization 
functions,  and  its  ability  to  recover  from  network 
disturbances. 

(c)  System  security  tests  -  To  verify  subnetwork  separation 
and  system  packet  integrity  and  protection,  and  to 
provide  a  preliminary  basis  for  security  certification. 

(d)  Network  availability  tests  -  To  demonstrate  that  the 
percentage  of  time  any  pair  of  users  are  able  to 
communicate  with  each  other  through  the  network  is  at 
least  99  percent  for  single-homed  subscribers  and  99.95 
percent  for  dual-homed  subscribers. 

All  five  phases  of  integration  testing  will  require  validation  of 
these  four  fundamental  DDN  test  objectives,  since  each  phase,  upon 
successful  completion,  will  produce  progressive  operational 
versions  that  must  provide  uniform  DDN  network  performance. 

Each  of  the  four  fundamental  objectives  has  a  corresponding  set  of 

testable  functions  which  will  be  demonstrated  for  DDN  validation. 

These  testable  functions  will  vary  somewhat,  however,  for  each  of 

the  five  integration  phases.  For  example,  security  testing  for 

the  MILNET  integration  will  not  be  as  extensive  as  security 

2 

testing  for  the  C  (WIN)  and  DoDIlS  integration.  Nevertheless, 


a  representative  set  of  these  functions  is  provided  in  Table  2-3. 
Tailoring  will  be  more  appropriately  defined  in  the  Test 
Procedures.  Table  2-3  also  shows  the  traceability  of  these 
required  functions  to  the  Program  Plan. 

Four  types  of  methodologies  can  be  employed  to  validate  the  DDN 
testable  functions  in  integration  testing.  These  include,  in 
their  usual  progression: 

(a)  Network  test  facility 

(b)  Intrasite  host  wraparounds/backbone  network  wraparounds 

(c)  Live  partitioned  network 

(d)  Live  network. 

The  network  test  facility  will  serve  two  purposes  during  DDN 
integration  testing.  It  can  simulate  a  partitioned  network  or  it 
can  be  configured  to  act  as  a  node  in  a  larger  network.  When 
configured  to  serve  as  a  simulated  network  with  no  connection  to 
the  actual  networks  that  make  up  the  DDN,  it  can  serve  as  an 
isolated  testbed  for  component  testing.  Configured  as  one  or  more 
nodes  on  the  DDN  or  a  subnetwork  of  the  DDN,  it  can  be  used  to 
test  component,  assembly,  or  subnetwork  functionality. 

Using  the  test  facility  in  these  ways  will  constitute  the  first 
step  in  DDN  integration  testing  and  allow  a  maximum  amount  of 
testing  with  minimal  risk. 

Intrasite  host  wraparounds  are  also  an  effective  methodology  for 
testing  and  validating  host  and  subnetwork  hardware  and  software. 
Host  wraparound  simulates,  within  the  host,  the  testable  functions 
of  all  DDN  network  activities,  including  responses  from  the 
network.  Using  this  capability  better  ensures  intrasite  integrity 
and  a  stable  platform  for  subsequent  network  test  and  analysis, 
and  allows  validation  of  the  host  software  and  much  of  the  test 
software  before  live  network  entry.  A  network  wraparound  can  also 
be  employed  to  ensure  proper  functioning  of  the  network  backbone. 


Table  2-3.  DDN  Testable  Functions  and 
Requirements  Traceability 


i 


a 

■tN 


£ 


I 

$ 


1 


y 


OBJECTIVE 

Network  Performance 
Parameters 


Network  Data  Handling 
and  Control 


Network  Security 


Network  Availability 
and  Reliability 


DDN 

PROGRAM 

TESTABLE  FUNCTION 

PLAN  SECTION 

Backbone  Delay 

4. 1.2/1. 2. 2 

Undetected  Bit  Error  Rate 

3. 2. 4/1. 2. 2 

Trunk  Transmission  Speeds 

2.9 

Node  Steady  State  Throughput 

2.1.2 

Node  Peak  Throughput 

2.1.2 

Misdelivery 

3.2.5 

Automatic  Diagnosis  and 

Recovery 

3.2.2 

Reconstitution 

3. 4. 1.4 

Preplanned  Rehoming 

3. 4. 1.5 

Reserved  Capacity 

3. 4. 1.6 

Capacity/Stress  Levels 

3.4.3 

Graceful  Degradation 

3. 4. 1.8 

Dynamic  Adaptive  Routing 

3. 4. 1.7 

Protective  Mechanisms 

3.2.3 

Precedence/ Preempt ion 

3.1 

Link  Encryption 

3.3.1 

Security  Level  Separation 

3.3.2 

Separation  of  Communities 

of  Interest 

3.3.3 

Individual  Access 

3.3.4 

Intercommunity  Operation 

3.3.6 

End-to-End  Encryption 

1.2.4 

3.2 

*1 
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Live  partitioned  networks  can  be  used  once  the  intrasite  and 
network  wraparounds  have  been  completed.  Any  limited  combination 
of  new  and  operational  DDN  sites  can  be  configured  to  demonstrate 
the  full  range  of  the  testable  DDN  functions  previously  outlined. 

Finally,  all  member  sites  interact  to  ensure  that  all  DDN 
functions  are  tested  and  perform  in  accordance  with  design 
specifications. 

2. 2. 3.1  Stages  I  and  II. 

2. 2. 3. 1.1  Hardware.  With  the  exception  of  production  IPLIs,  the 

Mobile  Reconstitution  Vans,  and  Blacker  technology,  all  DDN 

2 

equipment  should  be  integrated  on  the  separate  C  I  (WIN  and 

DoDI IS) ,  SECRET,  and  MILNET  subnetworks  in  this  period.  Hardware 

system  performance  functions  as  outlined  in  Table  2-3  can  be 

tested  on  the  subnetwork  level  to  ensure  continued  original 

performance  in  accordance  with  the  specifications  for  each.  Only 
2 

C  I,  however,  can  exercise  a  representative  range  of  DDN 
testable  functions  at  this  point,  and  can  do  this  only  on  a 
partitioned  network  basis  since  limited  prototype  IPLIs  will  be 
available.  Moreover,  tests  must  make  certain  that  the  required 
separation  of  data  is  maintained  across  all  subnetworks  of  DDN. 

2. 2. 3. 1.2  Software.  Most  of  the  DDN  software,  except  that 

required  for  the  hardware  noted  in  Paragraph  2. 2. 3. 1.1,  should  be 

2 

available  for  integration  testing  on  the  C  I,  SECRET,  and  MILNET 
subnetworks.  The  testable  software  functions  are  network  data 
handling  and  control  and  system  security,  as  detailed  in  Table 
2-3.  Again,  this  testing  will  deal  mainly  with  the  original 
individual  subnetwork  performance,  since  full  DDN  performance 
testing  can  only  be  accomplished  when  production  IPLIs  become 
available  in  Stage  III. 

2. 2. 3. 2  Stage  III. 

2. 2. 3. 2.1  Hardware.  When  production  IPLIs  are  available,  full 
DDN  hardware  integration  testing  can  proceed  and  will  be  phased  as 
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previously  defined.  The  testable  functions  that  must  be  validated 
for  each  of  the  remaining  four  phases  of  integration  are  defined 
in  Table  2-3  in  system  performance  parameters. 

2. 2. 3. 2. 2  Software.  Full  DDN  software  integration  testing  will 

proceed  in  accordance  with  the  testing  of  functions  outlined  in 

Table  2-3  for  network  data  handling  and  control  and  system 

security.  At  the  conclusion  of  successful  integration  testing  in 

2 

this  stage,  the  three  subnetworks  (C  I,  SECRET,  and  MILNET)  will 
operate  as  a  single  network  supporting  multiple  levels  of  security 
with  adaptive  routing.  This  will  establish  the  initial  operating 
capability  (IOC)  of  DDN. 

2. 2. 3. 3  Stage  IV. 

2. 2, 3. 3.1  Hardware.  Blacker  security  devices  made  available  in 
this  timeframe  will  require  the  same  DDN  integration  testing  as 
outlined  for  the  introduction  of  IPLIs  in  Paragraph  2. 2. 3. 2.1. 


2. 2. 3. 3. 2  Software.  Testing  requirements  for  integration  are  the 
same  as  those  outlined  for  IPLIs  in  Paragraph  2. 2. 3.2. 2. 

2,3  Description  of  Test  Ranking  Factors. 

2,3.1  T&E  and  IVV&T  Priority  Groups.  To  establish  priorities 
among  DDN  hardware  and  software  test  recommendations,  the 
following  five  groups  of  T&E  and  IVV&T  are  established: 

(a)  Priority  1  -  highest  priority  assigned  to  required 
testing.  Designates  those  tests  and  IW&T  activities 
determined  to  be  most  essential  to  development  and 
operation  of  the  DDN.  Includes  all  testing  necessary  to 
demonstrate  compliance  with  contractual  requirements. 

(b)  Priority  2  -  extremely  important  test  directed,  for 
example,  at  ensuring  the  integrity  of  the  system  in  terms 
of  service  to  the  user  as  well  as  control  of  the  network. 


(c)  Priority  3  -  test  and  validation  activities  that  provide 
added  assurance  that  objectives  will  be  met  and/or  are 
conducted  for  the  purpose  of  substantiating  previous  test 
results. 

(d)  Priority  4  -  test  and  validation  actions  performed  on 
support  equipment  not  directly  involved  in  network 
operation. 

(e)  Priority  5  -  ancillary  testing  directed  at  long-range 
improvements  and  upgrading  system  performance. 

2.3.2  Test  Histories  of  Hardware  and  Software  Items.  A  major 
factor  to  consider  in  rank  ordering  test  recommendations  is  the 
test  history  of  each  hardware  and  software  item  which  comprises 
the  DDN.  Testable  components  of  the  system  range  from  items  which 
have  a  long  operational  history  to  new  development  items  that  are 
not  yet  proven  operationally. 

Maturity  of  software  items  range  from  fully  developed 
off-the-shelf  packages  which  have  undergone  extensive  testing  and 
debugging  to  items  planned  for  development  under  Government 
contract  or  auspices  for  special  DDN  application. 

This  factor  for  establishing  priorities  focuses  on  the  stage  of 
development  of  the  hardware  and  software  items  and  depends  on  the 
availability  of  appropriate  test  documentation. 

2.3.3  Hardware  and  Software  Criticality  and  Complexity. 
Criticality  and  complexity  of  hardware  and  software  items  bear 
heavily  on  the  ranking  process.  Criticality  addresses  primarily 
the  functions  the  equipment  and  software  elements  are  required  to 
perform  and  involves  a  judgement  of  how  essential  each  element  is 
to  network  operation.  Assessment  of  complexity  focuses  on 
hardware  design  and  software  programming  aspects. 

2.3.4  Distribution  of  Hardware  and  Software  Items  Throughout  the 


Subnetworks.  The  extent  to  which  hardware  and  software  items  are 
used  throughout  the  subnetworks  affects  priorities  for  T&E  and 


IVV&T  program  recommendations.  A  high  incidence  of  item 
commonality  favorably  affects  the  test/benefit  ratio  and  supports 
a  modular  test  approach. 

2.3.5  Resources  Required  to  Accomplish  the  T&E  and  IVV&T 
Efforts.  The  DON  T&E  and  IW&T  program  recommendations  may  be 
influenced  by  the  amount  and  availability  of  resources  required  to 
accomplish  the  proposed  tests.  Similarly,  tests  which  require 
expending  considerable  resources  to  develop  a  new  test  facility  or 
modification  may  be  given  a  lower  priority  than  would  otherwise  be 
the  case  if  an  existing  test  facility  could  be  used  as  is  for  the 
testing. 

2.3.6  Sequencing  and  Scheduling  Considerations.  The  priority 
assigned  to  a  particular  test  or  IVV&T  recommendation  may  stem 
from  timing  considerations  as  well  as  program  dependencies  and 
sequencing  relationships.  Network  integration  milestones,  for 
example,  or  major  decision  points  may  dictate  test  priorities  at 
various  stages  of  DDN  implementation. 

Figure  3-1  shows  the  schedule  for  the  development  of  the  ddn, 
together  with  a  plot  of  the  growth  of  the  number  of  nodes  over 
time  and  a  plot  of  the  major  milestones  in  component  development. 
This  way  of  depicting  DDN  development  activity  indicates  when 
major  T&E  activity  must  take  place.  The  amount  of  node  site 
transition  testing  necessary  at  any  time  can  be  derived  from  the 
plot  of  node  growth.  Time  available  for  DDN  integration  testing 
can  be  obtained  from  the  plot  of  network  integration  activities. 
Component  testing  schedules  and  plans  will  have  to  be  developed 
after  consideration  of  the  component  development  schedules  shown 
in  the  lower  third  of  the  figure. 

2.4  Application  of  Test  Ranking  Factors. 


2,4.1  Component  Level  Testing.  In  assigning  priorities  to  the 
testing  of  system  elements  of  DDN,  the  two  major  considerations 
are  the  stage  of  development  of  the  element,  and  its  importance  to 
the  performance  of  system  functions.  If  an  element  or  a  major 
part  of  that  element  is  still  in  the  process  of  being  developed, 


then  it  should  require  more  testing  than  an  element  that  has 
already  proven  itself  in  the  field.  Likewise,  components  that  are 
essential  to  the  actual  utilization  of  the  DDN,  such  as  the 
network  access  software  and  the  IPLI  software,  must  be  considered 
more  important  than  such  supportive  elements  as  the  Power  and 
Environment  Monitor  (PEM)  or  the  Mobile  Reconstitution  Vans.  The 
priority  ranking  attempts  to  incorporate  these  distinctions  in 
criticality  to  network  performance  with  stage  of  development  to 
obtain  relative  levels  of  importance  for  testing  the  system 
elements  of  DDN. 

Table  2-4  lists  all  DDN  system  elements  (hardware  and  software) 
according  to  their  development  status,  criticality  to  network 
performance,  recommended  test  approaches  to  be  applied,  and 
priority  ascribed  to  that  testing.  The  importance  levels  are 
relative  only  to  the  elements  themselves,  with  1  being  the  highest 
level  of  importance  and  5  being  the  lowest. 

2.4.2  Site  Transition  Testing.  Site  transition  testing  is 
conducted  to  ensure  that: 


(a)  The  site  is  ready  for  cutover  to  DDN. 

(b)  The  site  host  subnetwork  is  not  affected  by  DDN 
components. 

(c)  DDN  components  are  not  adversely  affected  by  site  subnet 
equipment. 

For  these  reasons,  all  site  transition  testing  shown  in  Table  2-5 
is  considered  as  priority  1  except  for  the  establishment  of  the 
baseline  prior  to  DDN  upgrade.  Site  configuration  baseline 
testing  is  part  of  the  overall  data  base  required  for  documenting 
system  performance  before  DDN  access,  and  is  assigned  priority  2. 

2.4.3  Network  Integration  Testing.  Network  integration  testing 
should  carry  the  highest  priority.  It  is  imperative  that  testing 
at  this  level  fully  verify  that  DDN  provides  each  subnetwork 
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(C  I,  SECRET,  and  MILNET)  with  its  original  performance  quality 
and  services;  it  is  equally  important  to  demonstrate  the  required 
separation  and  security  of  data. 

Network  integration  testing  is  the  most  complex  and  critical  level 
for  DDN  hardware  and  software;  Test  history  will  be  limited  on 
the  network  level.  The  individual  network  test  objectives  of 
system  performance,  data  handling  and  control,  security, 
reliability,  and  availability  are  equally  significant  on  the 
network  level,  and  are  ranked  accordingly,  as  priority  1.  Table 
2-6  summarizes  DDN  integration  testing  and  indicates  this  priority 
1  ranking. 


3.  CONCLUSIONS 


3.1  Test  and  Evaluation  Ranking 


DDN  COMPONENT 
IPLI  Software 
NAC 

Mini-TAC 
Software 
HFEP  Software 
TEP  Software 
HID  Software 
Site  Preparation 


LEVEL  OF  TESTING 
Component 

Component 
Component 
Component 
Component 
Site  Transition 


Site  Facility  Site  Transition 

C/30  Switch  Node 
Trunk  Lines 
NAC 

Mini-TAC 

HFEP 

TEP 

HID 

Access-Line  Modems 
TAC 

Statistical  Multiplexers 

Switch  Level  Gates 

Mailbridges 

IPLI 

Blacker 

DDN  Network  Integration 


TYPE  OF  TEST  PRIORITY 

IVV&T  1 


IVV&T  1 
IVV&T  1 
IVV&T  1 
IVV&T  1 


Limited 

Operational 
for  Network 
Cutover  1 

Operational  1 


Operational  1 

(Performance , 

Data  Handling, 

Security,  Availability/ 
Reliability) 


LEVEL  OF  TESTING 
Component 


PRIORITY 
2 


DDN  COMPONENT 
Test  Facility 
Software 
Switch  Level 
Gate  Software 
Mailbridge 
Software 

Site  Preparation 

C/30  Software 
Network  Monitoring 
Center  Software 
IPL1  Hardware 


NAC 

Mini-TAC  Hardware 


HFEP  Hardware 


Component 
Component 
Site  Transition 


Component 

Component 


TYPE  OF  TEST 
Performance 
Validation 
Performance  2 

Validation 
Performance 

Validation  2 

Operational  2 

Existing  Performance 
Standards 

Quality  Assurance  3 

Quality  Assurance  3 


Component  Design  Verification/  3 

Reliability/ 

Environmental/ 

Maintainability/ 

Human  Engeering/ 
Electrical /Endurance 
Validation 


Component  Design  Verification/  3 

Reliability/ 

Environmental/ 

Maintainability/ 

Human  Engineering/ 

Electrical/Endurance 

Validation 

Component  Design  Verification/  3 

Reliability/ 

Environmental/ 

Maintainability/ 

Human  Engineering/ 
Electrical /Endurance 
Validation 


DDN  COMPONENT 
TEP 


HID  Hardware 


Test  Facility 
Hardware 

Switch  Level  Gate 
Hardware 


Mailbridge  Hardware 

Network  Monitoring 
Center  Equipment 
Hardware 


LEVEL  OF  TESTING 
Component 


Component 


Component 

Component 


Component 

Component 


TYPE  OF  TEST  PRIORITY 

Design  Verification/  3 
Reliability/ 

Environmental/ 

Maintainability/ 

Human  Engineering/ 

Elect rical/Endurance 
Validation 

Design  Verification/  3 
Reliability/ 

Environmental/ 

Maintainability/ 

Electrical /Endurance 
Validation 

Refer  to  Individual  3 
Elements 

Design  Verification/  3 
Reliability/ 

Environmental/ 

Maintainability/ 

Electrical/Endurance 

Validation 

Same  as  Switch  Level  3 
Gate  Hardware 
Reliability/  3 

Environmental 
Validation 


C/30  Hardware 


Component 


Reliability/ 

Environmental 

Validation 


3 


DDN  COMPONENT 
Trunk  Lines 


LEVEL  OF  TESTING 
Component 


TYPE  OF  TEST  PRIORITY 

Bit  Error  Analysis  3 


Hardware 

TAC 

Hardware 

Software 

Statistical 

Multiplexers 

Power  and 
Environment 
Monitor  (PEM)  - 
Hardware 
Software 

Patch  and  Test 
Modules  -  Hardware 

Software 

Access  Line  Modems 


ALRO  Hardware 


Software 

Mobile  Reconsti¬ 
tution  Vans 
Network  Information 
Center  (NIC) 
Hardware 
Software 


Component 

Component 

Component 

Component 

Component 

Component 

Component 

Component 
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3.2  Rationale  for  Ranking.  As  stated  in  Paragraph  2.4.1,  the  two 
major  considerations  in  ranking  components  should  be  the 
components'  development  status  and  importance  to  the  functioning 
of  the  DDN.  When  one  is  applying  these  two  variables,  component 
testing  can  be  subdivided  into  five  levels  of  importance  (see 
Paragraph  2.3.1).  The  network  access  approaches  and  1PLI,  because 
their  software  is  still  under  development  and  because  they  form 
the  basis  upon  which  the  whole  DDN  operates,  are  the  most  critical 
test  items.  The  software  associated  with  the  switch  level  gates, 
mailbridges,  and  test  facility  comprises  the  bulk  of  the  next 
level  of  testing  primarily  because  the  extent  of  software 
development  for  these  items  is  not  as  great  as  for  the  IPLI  or  the 
NAC.  The  Network  Monitoring  Center,  because  of  its  vital  role  in 
preserving  the  integrity  of  the  system,  has  also  been  assigned 
priority  2.  The  hardware  testing  of  the  access  approaches  forms 
the  basis  for  the  next  priority  level.  Its  criticality  to  network 
performance  is  the  same  as  its  software,  but  it  can  be 
differentiated  by  the  fact  that  some  of  the  hardware  already 
exists  with  established  performance  histories.  The  test  facility 
and  patch  and  test  modules  also  have  been  included  in  this  level 
because  of  their  supportive  nature.  The  last  two  levels  contain 
elements  having  functions  that  are  either  ancillary  to  the  DDN  or 
having  functional  histories  that  have  been  well  established. 

Site  transition  testing,  by  its  very  nature  of  preparing  the  user 
for  accessing  the  network,  is  a  vital  part  of  the  test  cycle. 
Therefore,  all  tests  for  site  testing  have  been  assigned  as 
priority  1  except  for  the  testing  of  baseline  host  performance 
standards,  which  has  been  assigned  as  priority  2. 

Integration  testing,  which  is  the  culmination  of  the  entire  test 
cycle,  is  the  most  critical  of  the  three  phases.  Therefore,  the 
network  tests  involving  performance,  data  handling,  security,  and 
availability  and  reliability  have  been  assigned  as  priority  1. 


3.2.1  Rationale  for  IVV&T  Ranking.  Independent  Validation, 
Verification,  and  Testing  (IVV&T)  is  used  when  the  Project 
Manager's  Office  determines  that  objective,  unbiased  monitoring 
and  testing  are  critical  to  the  development  of  a  particular 
software  component.  Because  IVV&T  is  an  expensive  form  of  testing 
in  the  short  run,  it  is  important  to  identify  those  factors  that 
make  IW&T  a  cost-effective  undertaking  in  the  long  run  before 
deciding  on  its  use. 

The  most  important  factors  in  a  decision  to  use  IW&T  are  the 
criticality  of  the  software  to  the  performance  of  the  system  that 
it  has  been  designed  to  support,  and  the  software's  complexity. 

In  cases  where  the  software  being  developed  is  critical  to  the 
performance  of  the  system  or  extremely  complex,  IVV&T  in  the  early 
development  stages  of  the  software  can  prevent: 

(a)  Cost  overruns 

(b)  Schedule  overruns 

(c)  Modifications 

(d)  Software  non-deliveries. 

Avoidance  of  these  can  serve  as  sufficient  justification  for  the 
cost  of  IVV&T. 

3.3  T&E  Location  Factors.  Generally,  hardware  and  software 
component  testing  should  be  done  at  the  manufacturer's  site. 

IW&T  may  be  conducted  at  the  manufacturer's  site  or  at  a 
Government-approved  test  facility.  TEMPEST  and  HEMP  testing 
should  be  accomplished  at  a  Government-approved  facility. 

As  the  software  for  a  piece  of  equipment  evolves,  stand-alone 
development  support  and  laboratory  subnetwork  testing  should  be 
accomplished  at  the  manufacturer's  site. 

Once  the  manufacturer  has  conducted  hardware/software  integration 
testing  of  a  configuration  item  (Cl),  it  should  be  turned  over  to 
a  government  test  facility  for  laboratory  environment  testing. 
Interfaces  between  CIs  should  also  be  tested  in  the  government 
facility. 


The  government  test  facility  containing  sufficient  network 
backbone  equipment,  interfaces,  and  circuits  will  be  able  to 
accept  equipment  from  all  vendors,  and  thus  can  more  closely 
emulate  the  operational  DDN  network.  Testing  in  this  manner 
provides  a  flexible  testing  environment  quickly  adaptable  to  DDN 
needs. 

Site  transition  testing  is  required  at  each  site,  since  no  two 
sites  are  exactly  the  same.  Further  inspection  and  evaluation  of 
sites  will  help  determine  the  amount  and  type  of  testing  required. 

Network  integration  testing  can  be  seen  as  two  distinct  types  of 
testing.  Network  integration  testing  in  the  test  facility 
provides  information  on  system  performance  and  capabilities  in  a 
controlled  test  environment.  This  testing  and  resultant 
information  increases  the  confidence  of  the  testor  (DDN)  and  the 
potential  subscriber  to  DDN  that  the  DDN  upgrades  or  changes  will 
not  degrade  an  operating  network. 

From  the  test  facility  environment,  network  integration  testing 
should  move  to  the  network  itself.  This  testing  should  involve 
partitioned  network  testing  of  functionality  before  cutover.  The 
partitioned  testing  uses  a  subset  of  the  tested  network  (including 
a  CONUS  terminal)  and  the  DDN  test  facility  as  nodes. 

3,4  Sequencing  and  Scheduling.  DDN  testing  will  progress  from 
the  component  level  to  site  transition,  and  then  through  network 
integration.  Test  requirements  are  outlined  in  the  previous 
corresponding  sections. 

The  sequencing  and  scheduling  of  individual  tests  required  at  the 
component  level  should  be  as  required  by  the  corresponding 
contract  and  will  be  further  defined  in  CDRLs  by  the  developing 
contractor.  Generally,  however,  as  long  as  the  contractor 
successfully  demonstrates  all  required  tests  within  its  allocated 
resources  and  delivers  the  equipment  on  time  to  support  site 
transition  testing,  a  specific  sequence  of  component  level  testing 
is  not  required. 


Site  transition  should  occur  as  soon  as  DON  production  equipment 
is  made  available  to  individual  sites.  The  particular  sequence 
will  depend  on  subscriber  acquisition  planning  and  funding 
availability.  Another  important  consideration  is  the  availability 
of  required  test  time,  which  will  vary  from  subnetwork  to 
subnetwork  and  site  to  site,  depending  on  operational  concerns. 
Specific  schedules  for  site  transitions  must  address  these 
considerations  and  should  be  derived  on  a  site-by-site  basis. 

Site  transition  testing  should  conclude  successfully  before 
network  integration  of  the  involved  site  begins. 

The  sequencing  for  DDN  integration  testing  should  follow  the  five 
progressive  phases  outlined  in  Paragraph  2.2.3.  Integration 
testing  of  all  the  subnetworks  should  conclude  by  the  end  of  Stage 
III  and  will  establish  the  initial  operating  capability  (IOC)  of 
DDN. 

Examination  of  the  schedule  of  activities  involved  in  the 
development  of  the  DDN  shows  that  site  transition  testing  may 
constitute  a  problem.  While  the  total  number  of  nodes  on  the 
network  is  not  now  certain  and  Figure  3-1  is  only  an  estimate,  the 
number  of  nodes  increases  significantly  in  the  year  before  the 
establishment  of  MILNET.  ARPANET,  MINET,  and  WIN  will  provide  50 
or  more  nodes  to  the  DDN  and  by  the  time  milnet  is  fully 
established,  or  shortly  thereafter,  the  DDN  will  have  perhaps  120 
nodes.  Since  a  realistic  estimate  of  the  time  required  to  test 
each  node  which  goes  through  this  transition  is  30  days,  careful 
planning  and  a  heavy  commitment  of  resources  will  be  necessary 
during  this  period. 
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4.  RECOMMENDATIONS 


4.1  Recommendations  for  Component  Level  Testing.  It  is 


recommended  that  the  component  level  testing  of  hardware  and 
software;  defined  in  detail  in  Paragraph  2.2.1,  be  conducted  with 
emphasis  on  the  software  and,  especially,  on  the  IVV&T.  If 
resources  of  time  and  funding  become  constraints  to  testing,  it  is 
recommended  that  the  component  level  tests  (summarized  in  Table 
2-4)  with  the  highest  priority  be  accomplished  first,  with 
subsequent  lower  priority  testing  accomplished  as  allowable. 

4.2  Recommendations  for  Site  Transition  Testing. 

4.2.1  A  coordinated  approach  to  site  transition  testing  is 
recommended.  This  approach  should  recognize  the  need  to  assure 
the  availability  of  network  services  at  the  switching  node  prior 
to,  or  in  conjunction  with,  subscriber  connectivity  testing. 

4.2.2  It  is  recommended  that  a  transition  plan  be  developed  for 
each  site.  These  plans  should  be  developed  3  to  6  months  before 
site  transition  testing  and  coordinated  with  all  participants. 

The  plans  should  include  items  such  as  method  of  DDN  network 
access  and  level  of  security  so  that  the  DDN  network  integration 
plan  may  incorporate  required  tests  for  overall  network 
integration. 

4.2.3  It  is  recommended  that  site  transition  testing  be  conducted 
consistent  with  the  DDN  evolution  schedule. 

4.3  Recommendations  for  Network  Integration  Testing. 


4.3.1  It  is  recommended  that  the  DDN  PMO  monitor  the  upgrading  or 
establishment  of  nets  and  sites  that  will  ultimately  become 
subscribers  to  DDN  in  order  to: 

(a)  Define  network  topology 

(b)  Determine  interface  requirements  to  DDN 

(c)  Facilitate  planning  and  coordination  of  a  comprehensive 
T&E  program  for  DDN  evolution. 
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4.3.2  It  is  recommended  that  the  DDN  test  facility  be  established 
and  brought  on-line  so  that  it  may  operate  as  a  part  of  any  net 
and  serve  as  the  point  of  insertion  for  functional  capabilities  of 
the  hetworks  to  be  integrated. 

4.3.3  It  is  recommended  that  a  Network  Integration  Plan  be 
developed  for  MINET  transition  to  the  MILNET  network.  This  plan 
should  be  ready  for  implementation  by  the  third  quarter  of  FY  84. 

4.3.4  It  is  recommended  that  a  Network  Integration  Plan  be 
developed  for  each  community-of-interest  network  transition  to 
DDN.  These  plans  should  be  ready  for  implementation  by  the  start 
of  FY  85. 

4.3.5  It  is  recommended  that  the  DDN  PMO  plan  to  conduct  network 
integration  tests  of  MINET  subscribers  to  the  DDN  backbone  during 
the  period  July  1984  to  January  1985. 

4.3.6  It  is  recommended  that  the  DDN  PMO  plan  to  conduct  network 
integration  tests  of  WIN/DoDIIS  subscribers  during  the  period 
October  1985  to  December  1986. 

4.3.7  It  is  recommended  that  the  DDN  PMO  plan  to  conduct 
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integration  testing  of  the  SACDIN  Network  into  the  C  I  Network 
during  the  period  January  1986  to  July  1986. 

4.3.8  It  is  recommended  that  the  DDN  PMO  plan  to  conduct 
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integration  testing  of  the  C  I  and  SECRET  networks  during  the 
period  April  1986  to  October  1986. 

4.3.9  It  is  recommended  that  the  integration  tests  of  the 
classified  and  unclassified  segments  of  the  DDN  be  conducted 
during  the  period  October  1986  to  April  1987.  This  testing 
follows  the  insertion  of  the  IPLI  into  the  network. 

4.3.10  Lastly,  it  is  recommended  that  a  comprehensive  DDN 
Transition  Plan  be  developed  in  consonance  with  the  subscribers  to 
identify  responsibility  for  network  control  during  each  phase  of 
DDN  development. 


LIST  OF  ABBREVIATIONS,  ACRONYMS,  AND  SYMBOLS 


ALRO 

ARPANET 

BBN 

CDRL 

Cl 

CONUS 

DCA 

DDN 

DoD 

DoDIIS 

DSARC 

DT&E 

EMI 

ESD 

FOT&E 

FY 

GFE 

KDH 

HDLC 

HEMP 

HFEP 

HID 

HIP 

IMP 

IOC 

IOT&E 

IP 

IPLI 

IW&T 

LMD 

MC 


Automatic  Line  Restoral  Option 
Advanced  Research  Projects  Agency  Network 
Bolt,  Beranek  &  Newman,  Inc. 

Contract  Data  Requirements  List 
Configuration  Item 
Continental  U.  S. 

Defense  Communications  Agency 

Defense  Data  Network 

Department  of  Defense 

DoD  Intelligence  Information  System 

Defense  Systems  Acquisition  Review  Council 

Developmental  Test  and  Evaluation 

Electromagnetic  Interference 

Electrostatic  Discharge 

Follow-on  Test  and  Evaluation 

Fiscal  Year 

Government-Furnished  Equipment 

High  Speed  Data  Link  Control  Distant  Host 

High  Speed  Data  Link  Control 

High-Altitude  Electromagnetic  Pulse 

Host  Front  End  Processor 

Host  Interface  Device 

Host  Interface  Protocol 

Interface  Message  Processor 

Initial  Operating  Capability 

Initial  Operational  Test  and  Evaluation 

Interface  Protocol 

Internet  Private  Line  Interface 

Independent  Verification,  Validation,  and  Test 

Lead  Military  Department 

Monitoring  Center 


LIST  OF  ABBREVIATIONS,  ACRONYMS,  AND  SYMBOLS  (Cont'd) 


MEP 

MILNET 

MINET 

mini-TAC 

NAC 

OT&E 

PAT&E 

PEM 

PMO 

PS 

QA 

SACDIN 

SOW 

T&E 

TAC 

TCP 

TEMP 

TEMPEST 

TEP 

WIN 


Management  Engineering  Plan 

Military  Network 

Movements  Information  Network 

Mini-Terminal  Access  Controller 

Network  Access  Component 

Operational  Test  and  Evaluation 

Production  Acceptance  Test  and  Evaluation 

Power  and  Environment  Monitor 

Program  Management  Office 

Packet-switching 

Quality  Assurance 

Strategic  Air  Command  Digital  Information  Network 

Statement  of  Work 

Test  and  Evaluation 

Terminal  Access  Controller 

Transmission  Control  Protocol 

Test  and  Evaluation  Master  Plan 

Certified  as  having  electronic  emission  controlled 
Terminal  Emulation  Processor 
WWMCCS  Intercomputer  Network 


